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1, INTRODUCTION 

Historically, the Electronic Visualization Lab was the first to create a data glove called Sayre Glove 
this was 1977 [1]. Thirty-five years later, the researchers adopted the camera to interact with the computer. 
In fact, the camera compared to the data glove is considered more direct and natural to achieve Human 
Computer Interaction [2]. 

Recently, the interactive by gesture has become widely used and in the future may replace the 
mouse and/or keyboard by vision-based devices. The main feature of using hand gestures is to interact with 
the computer as an input unit [3]. The gesture is defined as a form of nonverbal communication or non- vocal 
communication where the body's movement can convey certain messages. Gestures originate from different 
parts of the human body, but the most common ones emerge from the hand or face. 

Gesture provides a new form for interaction that reflects the experience of user in the actual world. 
The interaction by gesture be more natural and do not require any hindering or additional hardware. There are 
two kinds of gestures, static gestures and dynamic gestures. In [4] Liang introduced the best definition for 
hand posture or static gesture and dynamic hand gesture as: 

"Posture is a specific combination of hand position, orientation, and flexion observed at some time 
instance." 

“Gesture is a sequence of postures connected by motion over a short time span.’ 

The good examples of static hand gesture are "OK" or "STOP" and "No", "Yes", "goodbye" for 
dynamic gestures. 

Three approaches were used to obtain important information for any hand gesture recognition 
system, Data gloves approaches, Vision based approaches, and Color-Marker approaches as shown strictly in 
Figure 1. 
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In Vision based approaches [5, 6], the human motion obtained from one camera or more [7, 8], 
and the devices based on vision can handle many properties for interpreting the gesture, for example, color 
and texture whereas the sensor has not this property. Although these approaches are simple, many challenges 
can appear for example lighting diversity, complex background, and presence objects with skin-color similar 
to the hand (clutter), as well as the system, requires some criteria like time of recognition, speed, durability, 
and efficiency of computation [9-11]. 

Data glove approaches use types of sensors to capture the position and motion for the hand. In these 
approaches can compute in easy and accurate the coordinates of the locations of the fingers palm, and hand 
configurations [10-16]. The sensors do not achieve an easy connection with the computer because it needs to 
be the user connected physically with the computer and hinder the movement of the hand. These devices are 
also expensive, and unsuitable to operate in an environment of virtual reality [13]. According to Moore's 
Law, the sensors will become smaller and cheaper over time. It will prevalent in the future, we believe. 

Colored- Markers approaches used marked gloves worn by the hand of a human and be colored to 
help in the process of the hand tracking and to locate the fingers and palm. Marker gloves can form the shape 
of hand by extract the geometric-features [17]. In [18] used a wool glove with three different colors to 
represent the palms and fingers. This approach considers simple and not expensive if compared with Sensor 
or Data Glove [18], but the natural interaction between human and computer still not enough [17, 19]. 

However, there are some challenges when we want to design a strong and real-time gesture 
recognition system. Challenges related to complex hand structures that lead to difficulty in tracking and 
recognition. In addition, there are challenges related to the shape of the gesture, the different lighting 
conditions, the real time issue and the presence of noise in the background. These challenges taken into 
account in this paper by used running average principle in Background subtraction technique to detect and 
extract the hand from background and used contour of hand as feature. 


(a) “Data-Glove approaches" [12] (b) "Vision-Based approaches " (c)"Colored Marker approaches" 


Figure 1. Examples of input techniques for hand gesture recognition system 


There are many systems designed to recognize the hand gesture, we will mention to some of them. 
In [20] Amiraj and Vipul introduced a system to recognize the hand gestures for HCI. They used more than 
one approach for preprocessing step in their system and used two methods to perform segmentation process 
one with a static background and another without the constraint of background. In static background used 
constant value to a threshold, they used Otsu thresholding algorithm, and when the thresholding is dynamic 
used the color at the real-time mode. In the free background constraint, used thresholding methods based on 
color feature and subtraction of background model. For detecting the hand, they found the contour of it and 
then computing the convex hull and convexity defect to find the number of fingers. They provided three 
approaches to interact with devices, tracking of the finger, orientation of the hand, and counting of fingers. 

Shwetha k., Yewale et al. [21] provided a review for many hand gesture recognition systems that 
used Matlab language. They used a Canny edge algorithm to determine the edge of hand and used the values 
of hue and saturation for Skin —Color detection. They concluded that the system gave better results when 
used Artificial Neural Networks (ANNs) and edge detection methods. 

In [22], Nancy et al, introduced a system to hand gestures recognition by using Color-Marker 
approach. The user wearing white cloth in your hand and place a red color marker on hand's fingertip. 
The gestures in this system used to point on the computer's screen by detection the only finger with a red 
color marker. However, this system does not achieve direct contact with the devices because of it used the 
color marker. 

In [23] introduced a hand gestures recognition system by Tasnuva Ahmed. The system based on 
neural networks and in real time. The researcher divided the system into four steps, image capturing, 
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pre-processing, features extraction, and recognition. The system succeeded in distinguishing hand gestures 
that taking from different angles or size or orientations but there is a delay in the system due to the training 
phase of the Artificial Neural Network as well as delay in switching between the nodes. Badgujar et.al 
presented a dynamic hand gestures classification system using contour analysis. It is an efficient system for 
computer control, but it applies only to PowerPoint presentations [24]. 


2. RESEARCH METHOD 

The proposed system was coded by Python language, PyAutoGUI library for executing operations 
of mouse or keyboard, OS Module of Python and the OpenCV library. The general structure for any system 
to recognize the hand gestures can be explained as shown in Figure 2. Our system contains five steps, which 
are detailed. 


Captures image and 
forwards it to the system 
for processing 














Starts Webcam 


WEBCAM 


«s USER SYSTEM 
/» 


Processes the image and sends 
the result back to the User 


Figure 2. General structure of the hand gestures recognition system 


2.1. Image Capturing 
In this step used a webcam to acquire the RGB image (frame by frame) and based on only bare hand 
without glove or colored marker that can be hinder the user. 


2.2. Pre-Processing 

In this step in order to minimize the computation time we took only the important area instead of the 
whole frame from the video stream and this is called Region Of Interest (ROI). In image processing prefers 
to convert the color images into a grayscale images to increase the processing and after complete the 
processing can restore the images to its original color space, therefore, we convert region of interest into a 
grayscale image. Then blurring the (ROI) by Gaussian blur to reduce the objects that have high frequency but 
not the target. Notice that in this step the algorithm will fail if there is any vibration for the camera. 


2.3. Hand Region Segmentation 

This step 1s important in any system to hand gestures recognition and help in enhancing the 
performance of the system by removing the unwanted data in the video stream. In general, there are two 
methods to detect the hand in image, the first method depends on Skin-Color, this method is simple but 
effected by the light conditions in the environment and the nature of background. The second method does 
not depend on Skin-Color but on shape of hand and benefit from the principle of convexity in detection of the 
hand. The shape of hand is very important feature in the systems of recognition the hand gesture [25]. 

There are several methods or techniques used to extract the hand region from the image can be 
summarized as: 
a) Edge-Detection. 
b) RGB values because of the values of RGB for hand different from the background of the image. 
c) Subtraction of background 

In this paper used background subtraction technique to separate the hand from the background. 
This technique needs to determine the background that can be obtained by benefit from a running average 
principle. The background is computing from made the system focus on a certain scene for at least 30 frames 
and during that calculating the running average for the current frame and all previous frames using the 
following equation: 
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dst(x, y) = (1 - alpha).dst (x, y) + alpha.sre (x, y) (1) 


where, src (x,y) 1S a source image may be one or three channels and 8-bits or 32-bits floating point, dst (x,y) 
is destination image contains the same channels in the source image and 32-bits or 64-bits floating point. 
Finally, alpha is a weight of the source image and can be considered as threshold to determine the time for 
computing the running average over the frames. 

After determining the background, we put the hand front the camera, then compute the absolute 
difference between the background that calculates by using the running average and the current frame that 
contains the hand as a foreground object. Thus find the background and then compute the difference all that 
called subtraction background. 

The next step is thresholds the image that output from background subtraction and the result will be 
only hand with white color and the rest image with black color. The threshold process is important and must 
be done before the contours finding a process to achieve high accuracy. Mathematically can represent the 
threshold principle as follows: 


1, if x = threshold 
(x)= 


0, if x< threshold (2) 


where, f(x) is the intensity of the pixel. 

All the processes above called Motion-Detection. Figure 3 shown the output of Hand region 
segmentation process. Finally, perform a chain of morphological processes such as erosions and dilations to 
remove any small regions of noise. 





Figure 3. Output of hand region segmentation process 


2.4. Contour-Extraction 

The contour can be defined as object’s boundary or outline (hand in our case) that be located in the 
image. In other words, the contour is a curve connecting points that have the like color value and is a very 
important in shape analysis, objects detection and recognition process. 


2.5. Features Extraction and Recognition 

Now we turn to the second part of the research, which is how we determine the number of fingers. 
From the number of fingers can recognize the hand gestures, and for performing this task used Convex Hull 
to locate the extreme points (top, bottom, left and right). The convex hull is the group of points that 
surrounding the region of hand as shown in Figure 4. Here, we must clarify the principle of the Convex Set, 
which means all lines between any two points within hull are entirely within it. From extreme points can 
compute the palm's center, Figure 5 shown that. 

The next step is to draw a circle about the fingers, its center point is the center of the palm and the 
seventy percent from the length of maximum Euclidean distance between the palm’s center and extreme 
points represent the radius. After that using the bitwise AND operation between the circle from the previous 
step and the threshold image, the slices of fingers that result from this operation can use to compute the 
number of hand fingers. After determined the gesture based on the hand’s fingers number, the corresponding 
operation is performed. In fact, the process of distinguishing the hand gesture is a dynamic process. 
After performing the required instruction from the gesture, return to the first step to take another image to be 
processed and so on. 
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Figure 4. Convex hull Figure 5. Extreme points (top, bottom, left, right) and center of palm 


3. RESULTS AND DISCUSSION 

This paper recognized six gestures as shown in Figure 6. After recognition step can generate the 
command that associated with each gesture and passed to control the computer. The six gestures those 
recognized in this paper used to generate the actions of mouse and used to open various applications such as 
Microsoft Office (Word, Excel, PowerPoint), VLC Media Player, Google Chrome, Internet Explorer etc. 
The system implemented with different rotations, scale, lighting and plain or simple background. Table 1 
shows the experimental results of proposed system with threshold values (50,75) and alpha value 0.5. 
The performance of system is given in Chart |. The results of system shown that the rate of recognition is 
96.6%. The system was applied in different conditions, it has given the best results when the background is 
clear and the light is medium, the rate of recognition reached 100%. 





Figure 6. Shape of six gestures 


Table 1. Hand Gesture Recognition Rate 


Gesture name No.of input samples No.of recognized samples Recognition rate (%) Total recognition rate(%) 
Zero 10 9 90 
One 10 10 100 
Two 10 10 100 
Three 10 10 100 Oe 
Four 10 10 100 
Five 10 9 90 


Recognition rate (96.6%) 





Chart 1. Performance results 


A new hand gestures recognition system (Ahmed Kadem Hamed AlSaedi) 


54 O ISSN: 2502-4752 


4. CONCLUSION 

For a long time, the problem of distinguishing gestures was important in computer vision because of 
the challenge of extraction the target object such as hand from a background that has been cluttered and all 
that in real-time. In fact, the human when looking to a certain image can easy detecting what is inside it but 
that very difficult for the computer if looks to the same image because it deals with the image as a matrix 
with three dimensions. 

Our paper solved the challenge of rotation, orientation, scaling, and got the same results when used 
right or left hand. The system that provided used only bare hand and webcam of Laptop so it is very flexible 
for the user. The system does not need a database, but it is directly distinguishes the gesture and this achieves 
the speed of the system. The results of system shown that the rate of recognition is 96.6% and this result is 
considered very good compared with other research papers. In future and to enhance the system can use the 
both hands instead of use only right hand and that will increase the number of gestures. The results section 
showed that the best rate of recognition 1s when the background is clear and the light 1s medium, so these 
limits must be addressed in the future in order to increase the accuracy of the system. 
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